1. Field of the Invention
The present invention relates to a continuous data server apparatus and a data transfer scheme for realizing services of continuous data such as video data and speech data by responding to a plurality of access requests for the continuous data simultaneously, and making parallel accesses to a plurality of disk devices.
2. Description of the Background Art
A continuous data server apparatus for handling continuous data such as video data and speech data has functions for reading out the continuous data stored in memory devices, and transferring the data continuously in real time while synchronizing in time with terminal devices. Such a continuous data server apparatus is used in fields such as a video-on-demand in which video data for a plurality of movies, etc. are stored and an arbitrary movie is transferred according to a request from a terminal, and an on-line shopping in which information on commercial products is provided in video data through a network.
In such applications, the continuous data server apparatus will receive requests from a plurality of users at random, and therefore the continuous data server apparatus is required to have a capability to transfer respectively different continuous data to many different users simultaneously.
Now, various examples of such a conventional continuous data server apparatus will be described in detail.
An exemplary conventional continuous data server apparatus has a configuration as shown in FIG. 1.
In this conventional continuous data server apparatus of FIG. 1, an access request for the continuous data issued by a user or an application program will be sent through an inter-process communication, a communication via a network, etc. This access request is then notified from one of communication control devices 706 through a bus 720 to a central control device 710, and admitted by the central control device 710. The central control device 710 then notifies the reading of the requested continuous data to relevant data memory control devices 704, and these data memory control devices 704 read out the specified continuous data from data memory devices 702 and write them into a buffer memory device 708. The central control device 710 then commands the transfer of the data on the buffer memory device 708 to one of the communication control devices 706, and this communication control device 706 transfers the continuous data toward a transfer destination specified in the access request.
The central control device 710 basically comprises a CPU and a memory device, similarly as an electronic computer. The central control device 710 may be equipped with a communication control function for the purpose of notifying the access request, so that the access request from a user or an application program is notified by the communication control function of the central control device 710 rather than by the communication control device 706.
The data memory devices 702 for storing the continuous data are usually provided in forms of disk devices. For this disk device, a magnetic disk device is used in most cases, but there is also a case of using an optical disk or an opto-magnetic disk device, etc. Apart from the disk device, there is also a case of using a semiconductor memory device such as RAM or EEPROM for the data memory devices 702.
Now, in this conventional continuous data server apparatus of FIG. 1, which uses disk devices as the data memory devices 702, for example, there are cases in which accesses from many users are concentrated simultaneously to different portions of the continuous data stored in a particular disk device. For instance, this corresponds to a situation in which a particular video data is contained in a certain magnetic disk as the continuous data, and this particular video data is to be watched by many users at timings displaced from each other by one minute.
In such a case, in this conventional continuous data server apparatus of FIG. 1, the access speed of the disk becomes a bottleneck, and there is an inconvenience that it becomes impossible to transfer the data in real time to all the users. For example, in a case of dealing with video data, assuming that the continuous data are to be transferred at a rate of 1.5 Mbit/sec on average to a terminal of one user and that a magnetic disk device storing this video data can read out data at a rate of 2 MByte/sec at most, it would only be possible to realize a capability for transferring data from one magnetic disk device to ten users simultaneously at most.
As a technique for resolving such a problem associated with the disk device, a technique called striping is known. The striping is a technique for increasing an overall data transfer capability (total bandwidth) by using a plurality of disk devices, while making it possible for more numerous users to make accesses simultaneously to the identical continuous data by storing each one continuous data in division into a plurality of disk devices. By carrying out the striping, it becomes possible to construct a memory device with a wide bandwidth which can deal even with the worst possible case in which a plurality of users make accesses to different portions of the identical continuous data almost simultaneously.
In a case of using this striping technique with respect to n sets of disk devices, it is possible to take the exclusive OR of the data stored in these n sets of disk devices as a parity and store this parity in an (n+1)-th disk device, such that any data lost by a malfunction of one disk device among these (n+1) disk devices can be recovered by calculating the exclusive OR of the data in the remaining disk devices. This is a technique known as RAID (Redundant Arrays of Inexpensive Disks).
However, even when one central control device 710 is connected with numerous disk devices as shown in FIG. 1 so as to enhance the bandwidth by means of the striping, a number of users that can make accesses simultaneously will be limited by the transfer rate of the bus 720 associated with the central control device 710.
For example, in a case of using a PCI (Peripheral Component Interconnect) bus having a transfer capability of 133 MByte/sec, when the access speed for one piece of continuous data is assumed to be 1.5 MBit/sec, the data transfer will occupy total 3 Mbit/sec of the data transfer capability (bandwidth) of the bus 720 including 1.5 Mbit/sec used in transferring from the disk devices 702 to the buffer memory devices 708 per one piece of continuous data and 1.5 Mbit/sec used in transferring from the buffer memory devices 708 to the communication control device 706 per one piece of continuous data. Therefore, even if it is assumed that the bus is operated at 100% efficiency, it is going to be capable of making accesses to only up to 354 pieces of continuous data simultaneously. In practice, the bus will never be operated at 100% efficiency, so that a number of users that can make accesses simultaneously will be reduced further.
Thus, in this type of conventional continuous data server apparatus, there has been a problem that a number of users that can make accesses to the continuous data simultaneously in a single continuous data server apparatus is limited by the bandwidth of the data path such as the bus constituting the continuous data server apparatus.
For this reason, in order to realize the simultaneous accesses for so many users that cannot be realized by a single system of a conventional continuous data server apparatus, it is possible to consider a configuration shown in FIG. 2 in which a plurality of computers, each functioning as the continuous data server apparatus shown in FIG. 1, are provided in parallel. In this configuration of FIG. 2, the continuous data are stored by applying the striping among the data memory devices 802 connected to one continuous data server computer 800 (but the striping among the data memory devices 802 connected to different continuous data server computers 800 is not used in this example).
In this configuration of FIG. 2, a computer for overall control 801 receives the access requests for all the continuous data, and notifies each access request to the relevant continuous data server computer 800 having the requested continuous data. Then, the central control device (not shown) provided in the continuous data server computer 800 to which the access request is notified will control the transfer of the continuous data. According to this system of FIG. 2, it is possible to easily increase a number of users that can make accesses to the continuous data simultaneously by providing additional continuous data server computers 800.
However, unless a copy of every continuous data is provided in each group of the data memory devices 802 associated with each one of a plurality of the continuous data server computers 800, it will be impossible to deal with a case in which all the users make accesses to the different portions of the identical continuous data. Consequently, there has been a drawback in that it requires a considerable cost in providing copies of all the continuous data in all groups of the data memory devices 802 associated with many continuous data server computers 800.
Thus, in this type of conventional continuous data server apparatus, there has been a problem that a large cost is required for the data memory devices.
As another scheme for increasing a number of users that can make accesses simultaneously, which does not require copies of all the continuous data in many continuous data server apparatuses, there is a scheme using a configuration as shown in FIG. 3 in which data path from the data memory devices 902 to the communication control devices 906 is multiplexed by utilizing a massively parallel computer in which a plurality of central control devices (CPU) 910 and 911 are connected by a switch (coupling path) 921 with a large data transfer capability (bandwidth).
In this configuration of FIG. 3, memory devices 908 and 909 associated with the respective central control devices 910 and 911 are used as the buffer memory devices. Here, it is necessary for the switch 921 connecting between the central control devices 910 and 911 to be capable of handling the data transfer between the central control device 910 having an arbitrary data memory control device 904 and the central control device 911 having an arbitrary communication control device 906. For this reason, a multi-stage network such as the crossbar switch, the hypercube, or the ATM (Asynchronous Transfer Mode) network will be used for this switch 921.
However, in this continuous data server apparatus of FIG. 3, the data can be transferred from a plurality of central control devices 910 having the data memory control devices 904 to a single central control device 911 having a single communication control device 906, so that there is a need to provide a function of arbitration. Consequently, there has been a drawback in that the hardware of the crossbar switch or the ATM network, etc. constituting the coupling path 921 becomes complicated, and this also requires a considerable cost.
Thus, in this type of conventional continuous data server apparatus, there has been a problem that the hardware of the switch becomes complicated, and a large cost is requires for this switch.
On the other hand, there is another conventional continuous data server apparatus which has a configuration as shown in FIG. 4.
In this conventional continuous data server apparatus of FIG. 4, an access request for the continuous data issued by a user or an application program will be sent by means of an inter-process communication, a communication via a network, etc. to a central control device 610. The central control device 610 then notifies the reading of the requested continuous data to relevant data memory control devices 611, and these data memory control devices 611 read out the specified continuous data from data memory devices 612 and write them into a main memory device 613. The central control device 610 then commands the transfer of the data on the main memory device 613 to one of communication control devices 614, and this communication control device 614 transfers the continuous data toward a transfer destination specified in the access request. In a case the specified continuous data do not exist on the data memory devices 612, the specified continuous data are read out from an archiving device 615.
Similarly as in a case of FIG. 1, the data memory devices 612 for storing the continuous data are usually provided in forms of disk devices. For this disk device, a magnetic disk device is used in most cases, but there is also a case of using an optical disk or an opto-magnetic disk device, etc. Apart from the disk device, there is also a case of using a semiconductor memory device such as RAM or EEPROM for the data memory devices 612.
This conventional continuous data server apparatus of FIG. 4 is based on the known fact that it is possible to construct a system with a good cost performance by storing the continuous data in different types of memory devices according to their utilization frequencies.
Namely, the data with higher utilization frequencies are stored in data memory devices such as magnetic disk devices which require a relatively high recording cost per capacity but which enable a high speed access. Here, the access speed indicates a time required for a data cue search since the data request is received until the transfer of that data starts. By means of the on-line connection of these data memory devices, the data cue search time can be shortened.
It is also possible to use the data memory devices such as semiconductor memories which require an even higher recording cost but which enable an even faster access for those data which have particularly high utilization frequencies.
On the other hand, the data with lower utilization frequencies are recorded and managed by the archiving device. This archiving device is formed by memory devices such as magnetic tapes, optical disks, or opto-magnetic disks, which require a relatively low recording cost per capacity but which enable only a rather slow access. This type of memory device usually uses a tape or disk for recording the data which is removable from a drive. Namely, it has a structure in which the recording medium can be taken out from a drive for reading or writing the data. For this reason, the recording cost per capacity required in this type of memory device is only a cost of the recording medium itself, which is low compared with the magnetic disk device, etc. in which the recording medium is irremovable. However, in this type of memory device, when the data request is received, it is necessary to load an appropriate recording medium having the requested data onto a drive, so that the access speed is low for this type of memory device.
Thus, in this continuous data server apparatus of FIG. 4, the continuous data with high access frequencies are stored in the data memory devices 612 such as the magnetic disk devices, while the continuous data with low access frequencies are stored in the archiving device 615 such as the optical disks. When a request for the continuous data which are not stored in the magnetic disk devices is received, the requested continuous data are read out from the archiving device 615 to the data memory device 612.
Here, the continuous data read out from the archiving device 615 to the data memory device 612 can be transferred to the user either while the reading continues, or after the reading is completed. In the former scheme, a time required since the request from the user is received until the transfer of the continuous data starts is shorter, but the use of the visual search mode such as the fast forward mode will be restricted. In contrast, in the latter scheme, the time required since the request from the user is received until the transfer of the continuous data starts is longer, but the visual search mode such as the fast forward mode can be used freely because the continuous data are completely entered into the data memory device 612 by the time of the data transfer.
However, in this conventional continuous data server apparatus of FIG. 4, in a case of transferring the continuous data between the archiving device 615 and the data memory device 612, the continuous data will be temporarily written into the main memory device 613, so that there is a problem that a memory region of the main memory device 613 that is otherwise available for the transfer of the continuous data will be reduced in such a case.
In addition, in a case of transferring the continuous data between the archiving device 615 and the data memory device 612, the bus will be occupied at a time of transferring the continuous data between the archiving device 615 and the main memory device 613 as well as at a time of transferring the continuous data between the main memory device 613 and the data memory device 612, so that there is also a problem that a time for which the bus is otherwise available for the transfer of the continuous data will be reduced.
As a consequence, there is a problem that a number of continuous data (a number of streams) that can be transferred will be lowered while the continuous data are transferred between the archiving device 615 and the data memory device 612.
Moreover, for the same reason, there is also a problem that, when the continuous data read out from the archiving device 615 are to be written into a particular data memory device 612, a number of continuous data that can be read out from other data memory devices 612 and transferred will be limited.
In particular, in a case of applying the RAID technique to the data memory devices 612, there is a problem that a number of streams that can be transferred will be reduced after an occurrence of a malfunction of the data memory device, as it will become necessary to carry out the operation to repair the malfunctioned data memory device and write the data to be stored in that data memory device from the archiving device 615, while carrying out the operation to recover the continuous data stored in the malfunctioning data memory device by using the parity of the RAID technique and transfer the recovered continuous data.